Variational MCMC

We propose a new class of learning algorithms that combines variational approximation and Markov chain Monte Carlo (MCMC) simu­ lation. Naive algorithms that use the vari­ ational approximation as proposal distribu­ tion can perform poorly because this approx­ imation tends to underestimate the true vari­ ance and other features of the data. We solve this problem by introducing more so­ phisticated MCMC algorithms. One of these algorithms is a mixture of two MCMC ker­ nels: a random walk Metropolis kernel and a block Metropolis-Hastings (MH) kernel with a variational approximation as proposal dis­ tribution. The MH kernel allows one to lo­ cate regions of high probability efficiently. The Metropolis kernel allows us to explore the vicinity of these regions. This algorithm outperforms variational approximations be­ cause it yields slightly better estimates of the mean and considerably better estimates of higher moments, such as covariances. It also outperforms standard MCMC algorithms be­ cause it locates the regions of high proba­ bility quickly, thus speeding up convergence. We also present an adaptive MCMC algo­ rithm that iterates between improving the variational approximation and improving the MCMC approximation. We demonstrate the algorithms on the problem of Bayesian pa­ rameter estimation for logistic (sigmoid) be­ lief networks.